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Abstract 

Real social interactions occur on networks in which each individual is connected to 
some, but not all, of others. In social dilemma games with a fixed population size, hetero- 
geneity in the number of contacts per player is known to promote evolution of cooperation. 
Under a common assumption of positively biased payoff structure, well-connected players 
earn much by playing frequently, and cooperation once adopted by well-connected players 
is unbeatable and spreads to others. However, maintaining a social contact can be costly, 
which would prevent local payoffs from being positively biased. In replicator-type evolu- 
tionary dynamics, it is shown that even a relatively small participation cost extinguishes 
the merit of heterogeneous networks in terms of cooperation. In this situation, more con- 
nected players earn less so that they are no longer spreaders of cooperation. Instead, those 
with fewer contacts win and guide the evolution. The participation cost, or the baseline 
payoff, is irrelevant in homogeneous populations but is essential for evolutionary games 
on heterogeneous networks. 

Keywords: altruism, evolutionary game, complex networks, scale-free networks 
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1 Introduction 



A variety of creatures from many small organisms to social animals including humans show 
altruistic behavior. Altruism occurs even when being selfish is better for an individual, which 
constitutes the social dilemma (Dawes 1980; Sugden 1986). Emergence of altruism under social 
dilemmas can be explained by various mechanisms, such as kin selection, direct reciprocity, 
indirect reciprocity, and group selection (Nowak 2006). In the Prisoner's Dilemma, altruism is 
also promoted by the viscosity of populations (Axelrod 1984; Nowak & May 1992). If players 
are aligned on a spatially structured graph such as the square lattice, cooperators form close- 
knit clusters of conspecifics to survive the invasion of selfish defectors. Maintenance of such 
clusters is impossible in well-mixed populations modeled by the random graph and the all-to-all 
connected network. Spatial structure affects cooperation in other games as well (e.g. (Szabo & 
Hauert 2002)). Particularly, spatial structure can be detrimental to cooperation in the snowdrift 
game (Hauert 2002; Hauert & Doebeh, 2004). 

To be more realistic, players often inhabit networks of social contacts more complex than 
the square lattice, the random graph, and the all-to-all connected network (Newman 2003). 
First, real social networks are small-world, implying the combination of abundant localized 
interactions, as in the square lattice, and sufficient shortcuts, as in the random graph. Second, 
players are heterogeneous in terms of the number of contacts with others. An extreme case 
of this is the scale-free network in which the number of neighbors is distributed according to 
the power law (Barabasi & Albert 1999). In contrast, the number of neighbors of conventional 
networks is the same for everybody (regular lattices and the all-to-all connected network) or 
distributed with a narrow tail (so-called Erdos-Renyi random graph). Even though not all 
social networks are scale-free, the number of neighbors is naturally heterogeneous. 

Recently, it was shown that heterogeneous networks promote evolution of cooperation in 
the Prisoner's Dilemma, the snowdrift game, and the stag hunt game. Particularly, scale-free 
networks are strong amplifiers of altruism (Duran & Mulct 2005; Santos & Pacheco 2005; Santos 
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& Pachcco 2006; Santos et al. 2006a; Santos et al. 2006b). 

To explain the mechanism of enhanced cooperation, let us introduce the notion of the 
temperature of players. In evolutionary graph theory, hot players are those replaced often 
by others (Lieberman et al. 2005). By modifying this definition shghtly, we denote by hot 
players those who play often, namely, players with many neighbors. Cold players are those 
with a small number of neighbors, such as leaves in a network. Hot players arc allowed in more 
rounds of the game than cold players per generation. If the typical payoff obtained by playing 
a game is positively biased, which is a common assumption adopted in the previous studies 
cited above and others (Abramson & Kuperman 2001; Ebel & Bornholdt 2002; Ifti et al. 2004; 
Zimmermann & Egufluz 2005; Egufluz et al. 2005; Santos et al. 2006c) (but sec (Pacheco et 
al. 2006; Tomassini et al. 2006) for other setups), it is largely worth participation. Then hot 
players earn more than cold players because everybody earns positive 'base' payoffs proportional 
to the number of neighbors. As a result, hot players are more successful in disseminating their 
strategies. Particularly, cooperation once employed by a hub is stable due to cooperative 
reactions in its neighborhood. In addition, if hubs tend to be connected to each other, as in the 
model proposed by Barabasi and Albert (BA model) (Barabasi & Albert 1999), coopcrators on 
hubs form loose clusters to take advantage of a variant of spatial reciprocity (Santos & Pacheco 
2005; Santos et al. 2006a; Santos et al. 2006b). 

However, hot players are successful not owing to playing well but owing to the connectivity. 
In the present paper, we critically re-examine the effect of heterogeneous networks on emergence 
of cooperation. In real lives, participation in the game may be costly. A link to a neighbor 
implies building and maintaining communication, and this cost has actually been modeled for 
studying network formation (Jackson & Wolinsky 1996; Bala & Goyal 2000; Goyal & Vega- 
Redondo 2005). Expensive entry fees would dismiss the premium of hot players to change the 
entire scenario. 

We study two-person games on networks with participation costs. For simplicity, networks 
are assumed to be fixed in size and topology. We show that there are three regimes depending on 
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how costly participation is. First, when participation is inexpensive, cooperation is enhanced 
on heterogeneous networks as discovered previously. Second, when the participation cost is 
intermediate, the effect of the local payoff structure, namely, the configuration of the two-person 
game, and that of the network are comparable. Then altruism does not develop. Third, when 
participation is very costly, initial strategies of cold players have long transients and propagate 
to hot players. With small and large participation costs, the network rather than the local 
payoff structure determines evolutionary dynamics. In the intermediate regime, evolution is 
most sensitive to the local payoffs. 

2 Model 

We compare effects of two types of networks on the evolution of cooperation by means of Monte 
Carlo simulations. A diluted well-mixed population is modeled by the regular random graph 
in which each player has 8 neighbors that arc chosen randomly from the population. The 
heterogeneous networks are modeled by scale-free networks generated by the BA model, in 
which the number of neighbors denoted by k follows the power law p{k) oc (Barabasi & 
Albert 1999). The average number of neighbors in the scale- free networks is set equal to 8 for 
fair comparison with the regular random graph. Both types of networks consist oi n — 5000 
players. 

To probe the network effect, we consider only two simple strategies without memory, namely, 
unconditional cooperation and unconditional defection. The initial fraction of cooperators is 
set equal to 0.5. In one generation, everybody participates in the two-person game with all the 
neighbors. The payoff matrix will be specified in the next section. 

Each player tends to copy successful strategies in its neighborhood. We apply the update rule 
compatible with the replicator dynamics, following the previous literature (Santos & Pacheco 
2005; Santos et al. 2006a; Santos et al. 2006b). Suppose that player x with kx neighbors 
has obtained generation-payoff Px- To update the strategy, x selects a player y among the 
kx neighbors with equal probability (= l/kx). Then x copies y's strategy with probability 
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(Py — P.J.) / {max {k^i ky) ■ [uppermost payoff in one game - lowermost payoff in one game] } if 
Py > Px- The denominator is the normahzation constant so that the replacement probability 
ranges between and 1. If Pj^ < P^-, the strategy of x is unchanged. All the players experience 
updating according to this rule synchronously. This completes one generation. 

Each evolutionary simulation consists of 5000 generations. The final fraction of cooperators 
denoted by c/ is calculated as the average fraction of cooperators based on the last 200 gen- 
erations of 5 runs with different initializations for each network and 5 different realizations of 
the network. When the participation cost is not very large, c/ corresponds to values close to 
stochastic stationary values. Otherwise, c/ represents transient values. 

3 Results 

3.1 Prisoner's Dilemma 

We start with two well-known payoff matrices of the simplified Prisoner's Dilemma. The first 
one is given by 

C D 
C ( 1 
D\T 

The entries of Eq. ([T]) indicate the payoff that the row player gains when playing against 
the column player. The first (second) row and column correspond to cooperation (defection). 
The Prisoner's Dilemma arises when T > 1, and larger T results in more defectors. With 
participation cost h, the payoff matrix becomes 



(1) 



1-h -h 
T-h -h 



(2) 



Note that introducing h does not trespass the notion of the Prisoner's Dilemma as far as T > 1. 

Actually, the game defined by Eq. ([T]) or Eq. lies on the boundary between the snowdrift 
game analyzed in Sec. 13.21 and the Prisoner's Dilemma. Therefore we also examine another 
standard payoff matrix of the generic Prisoner's Dilemma given by 
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C D 
C (b-c -c 
D\b 



(3) 



where b > c. With the participation cost, Eq. is transformed to 



1 — r — /i — r — h 
1-h -h 



) 



(4) 



where r = c/b. In the following, we refer to numerical results for the payoff matrix given by 
Eq. ([2]). The results that are qualitatively the same as the following are obtained for Eq. (jlj), 
as shown in Supplementary figure [I](b-d) . 

The fraction of cooperators c/ is not affected by h on the regular random graph (Supple- 
mentary figure [it^a)). Because each player has the same number of neighbors, participation 
cost does not differentiate the players. In contrast, h drastically affects Cf for the scale-free 
networks, as shown in Fig. [Tt^a). In Fig. [T](b), cj for the scale-free networks relative to cj for the 
regular random graph is plotted. We identify three qualitatively different regimes in terms of 
h, which roughly correspond to (I) h < 0.24, (II) 0.24 < h < 2, and (III) h > 2. The transition 
between (II) and (III) is fairly gradual. 

3.1.1 Regime (I): Costless Participation 

When h < 0.24, participation is inexpensive, and hot players such as hubs are strong com- 
petitors regardless of the strategies of their cold neighbors. The payoff of a player increases 
linearly with the number of contacts, as shown by thin lines with positive slopes in Fig. [21(a). In 
particular, cooperation spreads from hot cooperators to their cold neighbors, the local density 
of cooperators increases, and hot cooperators gain more by mutual cooperation. Cooperation 
triggered by hot players is self-promotive. Defective hot players may also win for a moment. 
However, defection then prevails in their neighborhood so that hot defectors can no longer 
exploit the neighbors because of mutual defection. This results in a null generation-payoff of 
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hot defectors so that they can be outperformed by their cold neighbors. A hot player sticks to 
cooperation but not to defection. 

In sum, heterogeneous networks enhance altruism, which recovers the previous work cor- 
responding to h = (Santos & Pacheco 2005; Santos & Pacheco 2006; Santos et al. 2006a; 
Santos et al. 2006b). Note that this regime extends to /i < 0, that is, when gifts are given for 
participation so that everyone wins a positive payoff. 

To illuminate on different dynamics of hot and cold players, we measure how often players 
flip the strategy. The average number of flips throughout the evolutionary run (including the 
contribution from both transients and stationary states) is shown in Fig. [2](b). Colder players 
experience more flips when h < 0.24 (thin lines). They myopically follow what hotter players 
do both in transients and in stationary states. Cooperation on hubs is stabilized in an early 
stage, yielding less flips of hotter players. 

3.1.2 Regime (II): Moderately Expensive Participation 

Interestingly, cold players spread their strategies to hot players when h > 0.24 (regimes (II) 
and (III)), which is opposite to what happens in regime (I). As a result, enhanced cooperation 
diminishes, even with a relatively small participation cost. 

Regime (II) is defined by small to intermediate h (0.24 < h < 2). Now the local payoff 
structure as well as the network topology is relevant. When h = 0.3, scale-free networks surpass 
the regular random graph in terms of the number of cooperators only for 1 < T < 1.4. When 
h = 0.6, this range shrinks to 1 < T < 1.1. The privilege of scale-free networks is entirely 
lost when h = 1. In regime (II), the payoff of a player linearly decreases with the number of 
contacts (thick lines with negative slopes in Fig. [21^a)). Consistent with this, hot players flip 
strategies more often than cold players (thick lines in Fig. EJ^b)). Hubs no longer conduct the 
dynamics. 
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3.1.3 Regime (III): Costly Participation 

When h is roughly greater than 2, participation is really costly. Then cold players with any 
strategy surpass hot players and govern the dynamics. This is not because cold players are 
tactical but because they play less often and lose less than hot players do. Strategies of cold 
players are not often replaced owing to the small number of neighbors. Consequently, cold 
players persist in their initial strategies, actually up to > 300000 generations. In the meantime, 
strategies of cold players tend to spread to hot neighbors. As a result, cj, which is measured 
at 5000 generations, is almost the same as the initial fraction of cooperators (= 0.5) regardless 
of T (Fig. [11(a)). Figure [11(b) indicates that c/ is larger for the scale-free networks than for 
the random graph, but this is due to long transients. After a very long period, the fraction of 
cooperators will be smaller than c/ shown in Fig. [11(a) to eradicate the spurious advantage of 
scale-free networks. 

3.2 Snowdrift Game 

The snowdrift game, which is also known as the chicken game or the hawk-dove game (Sugden 
1986; Hauert & Doebeli, 2004), is illustrated as a situation of two drivers caught in a snowdrift. 
For the two cars to get out, which is equivalent to payoff (3 {morethanl) for each driver, the 
snow must be shoveled away. A total effort of unity must be invested to this task. Two players 
may cooperate to share the cost so that each pays 1/2, or one player may cover the full cost. 
Otherwise, both players miss (3. Cooperation benefits not only the opponent but also the focal 
player itself. Different from the Prisoner's Dilemma in which defection is beneficial, in the 
snowdrift game, cooperation can deserve even when the opponent defects. 

If the participation is free, the payoff matrix of the snowdrift game is given by 



In this case, heterogeneous networks reinforce evolution of cooperation as in the Prisoner's 



C 

C ( (3-1/2 
D\ (5 




D 



(5) 
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Dilemma (Santos & Pacheco 2005; Santos et al. 2006a; Santos et al. 2006b). Without dismiss- 
ing the structure of the snowdrift game, the participation cost translates the payoff matrix to 
the following form: 

(6) 



(3-ll2-h (3-1-h 
13 -h -h 



As shown in Supplementary figure [21 the participation cost h does not influence c/ on the 
regular random graph. The fraction of cooperators converges to a value close to the theoretical 
estimate c/ = 1 — r, where r = 1/(2/? — 1) is the cost-to-benefit ratio (Sugden 1986; Hofbauer 
& Sigmund 1998; Hauert & Doebeli, 2004). If cooperation is relatively costly with a small /3 
(large r), cooperators decrease in number. 

On heterogeneous networks, how the fraction of cooperators varies as a function of r depends 
on the participation cost. We again find three regimes as shown in Fig. [3l The scale-free 
networks host more cooperators than the regular random graph only when h is near zero or 
negative (regime (I)). The advantage of the scale- free networks is neutralized by intermediate 
h (roughly speaking, h = l), which defines regime (II). Note that the reduction of cooperation 
is not as much as that for the Prisoner's Dilemma. With large h (roughly speaking, h > 2), Cf 
is rather insensitive to the local payoff structure due to long transients (regime (HI)). 

3.3 General Two-person Games 

With the participation cost incorporated, general symmetrical two-person games with two 
strategies are represented by 



C 
D 



C D 

I 



(7) 



R-h S-h 
\T-h P-h 

In accordance with the previous sections, we denote by cooperation (defection) the strategy 
corresponding to the first (second) row and column. As T increases, players are tempted to 
defect, and Cf decreases. As S decreases, players would refuse cooperation to avoid exploitation 
by defectors. Therefore, Cf decreases. The Prisoner's Dilemma, the snowdrift game, and the 
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stag hunt game, are defined by T>R>P>S,T>R>S>P, and R > T > P > S, 
respectively. The Prisoner's Dilemma usually accompanies another condition 2R > T + S so 
that mutual cooperation is more beneficial than alternating unilateral cooperation. 

Multiplying each element of Eq. ([7]) by a common constant modifies just the time scale of 
evolution. Accordingly, there are three free parameters in the payoff matrix, which are chosen 
to be T, S, and h, while we set i? = 1 and P = 0. 

In Fig. m^a), c/ for h = is plotted in the T-S parameter space for the regular random 
graph. As expected, the number of cooperators decreases with T and increases with S. The 
results are independent of h. For the scale-free networks, we plot c/ in Fig.|l](b-e) for four values 
of h (also see Supplementary figure [3] for the direct comparison of c/ for the scale-free networks 
and Cf for the regular random graph). We confirm the existence of the three regimes, extending 
the results shown in Figs. [T] and O First, as shown in Fig. Hfb), scale-free networks promote 
evolution of cooperation when the participation is costless (Santos et al. 2006a). Cooperation 
is strengthened in the Prisoner's Dilemma (T > 1, 5* < 0), the snowdrift game (T > 1, S* > 0), 
and also the stag hunt game (T < 1, S* < 0). Second, the advantage of the heterogeneity is 
lost for a wide range of T and S when h = 0.5 (Fig. 111(c)) and h = 1 (Fig. 111(d)). Third, the 
transient is very long when participation is costly {h = 2). This allows defectors to survive 
for a long time even without dilemma (5* > 0,T < 1) and considerable cooperators to survive 
under the Prisoner's Dilemma (Fig. Hl^e)). 

4 Discussion 

We have discovered that the participation cost, which is irrelevant in well-mixed populations 
and on homogeneous networks including the regular lattices, casts a dramatic effect on evolu- 
tionary dynamics on heterogeneous networks. When participation is nearly free (regime (I)), 
heterogeneous networks promote cooperation (Duran & Mulct 2005; Santos & Pacheco 2005; 
Santos & Pacheco 2006; Santos et al. 2006a; Santos et al. 2006b). This is because the coop- 
eration on hot players is robust against invasion of whatever strategies of cold players. Even 
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if a cold player is good at exploiting cooperators in the neighborhood, the aggregated payoff 
would be much smaller than that of a hot player that would earn a lot just by playing the 
game many times. Hot players are leaders, and cold players are myopic followers. However, 
this phenomenon is not robust against variation in participation costs, which is consistent with 
the loss of cooperation under positive affine transformation of the payoff matrix (Tomassini 
et al. 2006). When the participation cost is intermediate (regime (II)), cooperators do not 
really increase and even decrease on heterogeneous networks. When participation is costly 
(regime (HI)), hot players myopically follow cold players. In regimes (I) and (HI), not local 
payoff structure but network structure governs evolution. The local payoffs are relevant only 
in regime (II), for which cooperation is not enhanced by heterogeneous networks. Regimes (II) 
and (HI), which have been largely unexplored, may be relevant in, for example, environmental 
problems, political conflicts, and human relationships. In these situations, players are often 
forced to play and incur participation costs. In other words, the best one could get may be the 
least disastrous, but not really wonderful, solution. 

For replicator-type and many other update rules in well-mixed populations, evolution is 
invariant under uniform addition of a constant to the payoff matrix (e.g. participation cost). 
Then a game in regimes (II) and (HI) can be translated into a game in regime (I). However, 
this operation is disallowed for heterogeneous networks. Because multiplying the payoff matrix 
by a positive constant alters just the evolutionary time scale in either case, there are two free 
parameters for two-person games in homogeneous populations, whereas there are three of them 
in heterogeneous populations (e.g. S", T, and h as used in Fig. H]). 

The present results can be generalized in some aspects. First, two-person games can be 
asymmetrical. Second, the update rule does not have to be of a replicator type (Hofbauer & 
Sigmund 1998; Ohtsuki et al. 2006) as far as the reproduction rate is not extremely nonlinear 
in the generation payoff. Third, large noise would blur but would not break down the three 
regimes. For example, irrational actions may cause hot cooperators, which are stable with 
small participation costs and small noise, to defect and elicit upsurges of defectors nearby. 
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However, noise does not overturn the fact that hot (cold) players are better off for small (large) 
participation costs. Fourth, network models can be arbitrary. For example, the scale-free 
networks based on the configuration model (without growth and preferential attachment) and 
the Erdos-Renyi random graph promote altruism in regime (I), albeit to a lesser extent than 
the BA model (Santos & Pacheco 2005; Santos et al. 2006a; Santos et al. 2006b). These 
heterogeneous networks as well as the BA model do not enhance altruism in regimes (II) and 
(III) (data not shown). 

The present results also have limitations. First, we have only assumed memoryless strategies, 
namely, unconditional cooperators and unconditional defectors. Second, the networks have been 
static. In coevolutionary dynamics in which players form and sever links as well as play games, 
only regime (I) has been considered, a main conclusion being enhanced cooperation (Skyrms 
& Pemantle 2000; Zimmermann & Egufluz 2005; Eguiluz et al. 2005; Santos et al. 2006c). 
In regime (III) , for example, everybody must sever hnks to be loners (Goyal & Vega-Redondo 
2005) (for the role of loners, also refer to (Hauert et al. 2002)). Third, we have used the 
additive payoff (Nowak et al. 1994; Abramson & Kuperman 2001; Ebel & Bornholdt 2002; 
Ifti et al. 2004; Santos & Pacheco 2005; Santos & Pacheco 2006; Duran & Mulct 2005; Santos 
et al. 2006a; Santos et al. 2006b). An alternative is to use the average payoff, or division of 
the generation payoff of each player by the number of neighbors (Kim et al. 2002; Santos & 
Pacheco 2006; Taylor & Nowak 2006). The average payoff is not affected by the number of 
neighbors and hampers the enhanced altruism on heterogeneous networks (Santos & Pacheco 
2006; Tomassini et al. 2006). We have adopted the additive payoff scheme because its intuitive 
meaning is clearer than the average payoff. These topics warrant for future work. 
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Figure captions 

Figure 1: The Prisoner's Dilemma on the scale-free networks with participation costs. The 
payoff matrix is represented by Eq. ([2]). (a) The final fraction of cooperators c/. (b) c/ for the 
scale-free networks shown in (a) minus C/ for the regular random graph shown in Supplementary 
figure [1^. 

Figure 2: The effect of the number of neighbors in the Prisoner's Dilemma on the scale-free 
networks, (a) The average generation payoff and (b) the average number of flips are plotted 
in terms of the number of neighbors that a player has. Note that the log scale is used for the 
abscissa of (b) for clarity. The lines correspond to h = (thinnest line), 0.2, 0.23, 0.24, 0.25, 
0.3, and 0.5 (thickest line). The payoff matrix is given by Eq. ([2]) with T = 1.5. 

Figure 3: The snowdrift game on the scale-free networks with participation costs, (a) Cf, 
and (b) Cj for the scale-free networks shown in (a) minus Cf for the regular random graph shown 
in Supplementary figure [2l 

Figure 4: c/ in the T-S space for (a) the regular random graph and the scale-free networks 
with (b) h = 0,{c)h = 0.5, (d) h = I, and (e) h = 2. 
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Supplementary figure 1: (a) The final fraction of cooperators Cf in the Prisoner's Dilemma 
on the regular random graph. The payoff matrix is represented by Eq. ([2]). The results for 
the payoff matrix Eq. (jlj) are shown in (b-d). (b) Cf for the regular random graph, (c) Cf for 
the scale- free networks, and (d) Cf for the scale-free networks minus Cf for the regular random 
graph. 

Supplementary figure 2: c/ for the snowdrift game on the regular random graph. 

Supplementary figure 3: Cf for the scale-free networks (Fig. |l](b-e)) minus Cf for the regular 
random graph (Fig. Ht^a)) in the T-S space, (a) h = 0, {h) h = 0.5, (c) h = 1, and (d) h = 2. 
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